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FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  is  concerned 
with  the  demands  of  the  future  battlefield  for  increased  man-machine 
complexity  to  acquire,  transmit,  process,  disseminate,  and  utilize  infor- 
mation. The  research  focuses  on  the  interface  problems  and  interactions 
within  command  and  control  centers  and  deals  with  such  areas  as  topo- 
graphic products  and  procedures,  tactical  symbology,  information  manage- 
ment, user-oriented  systems,  staff  operations  and  procedures,  and  sensor 
systems  integration  and  utilization. 

One  area  of  special  interest  is  the  extraction  of  information  from 
surveillance  displays  and  the  efficient  processing  of  this  information 
within  advanced  automated  image  interpretation  facilities.  Research 
results  are  used  in  systems  requirements  determinations  and  in  the 
development,  evaluation,  and  application  of  doctrine,  operating  proce- 
dures, and  system  design  concepts.  The  present  publication  is  the  sec- 
ond study  in  a series  exploring  instructional  techniques  for  more 
efficient  training  of  image  interpreters  in  target  identification.  The 
first  study,  ARI  Technical  Paper  209,  indicated  the  feasibility  of 
computer-assisted  instruction  to  maintain  and  improve  interpreter  skills, 
using  existing  computers  within  an  automated  image  interpretation  facil- 
ity. The  present  study  indicates  that  effective  training  in  target 
identification  can  be  provided  with  minimum  instructor  participation  and 
relatively  simple  support,  using  operational  imagery  as  the  instructional 
material. 

Research  in  the  area  of  sensor  systems  integration  and  utilization 
is  conducted  as  an  in-house  effort  augmented  by  contracts  with  organiza- 
tions selected  as  having  unique  capabilities  and  facilities  for  research 
on  sensor  systems.  The  present  study  was  conducted  by  personnel  from 
System  Development  Corporation  under  contract  DAHC  19-67-C-0040  under  the 
program  direction  of  Abraham  H.  Birnbaum.  The  effort  was  responsive  to 
requirements  of  Army  Project  2Q662704A721  and  to  special  requirements  of 
the  U.S.  Army  Intelligence  Center  and  School,  Fort  Huachuca,  Ariz.,  and 
the  Assistant  Chief  of  Staff  for  Intelligence. 
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EVALUATION  OF  FOUR  TARGET- IDENTIFICATION  TRAINING  TECHNIQUES 


BRIEF 


I Requirement : 

To  develop  a flexible  instruction  technique,  using  the  equipment 
to  be  found  within  an  advanced  computer-based  image  interpretation 
[ facility,  to  train  interpreters  in  visual  discrimination  of  fine  detail. 

i 

\ 

^ Procedure : 

I 

Alternative  experimental  training  techniques  for  maintaining  or 
improving  image  interpreter  performance  in  an  operational  facility  were 
evaluated  across  two  types  of  target  (tank  and  truck) . For  each  type  of 
target,  one  of  four  instructional  methods  was  administered  to  8 of  32 
recent  graduates  of  the  U.S.  Army  Intelligence  School.  Two  of  the  four 
methods  used  pictures  instead  of  text,  one  in  a random  presentation  and 
the  other  in  a structured  sequence  of  increasing  difficulty.  The  third 
method  used  programed  text  to  teach  verbal  identification  cues,  and  the 
fourth  combined  programed  text  in  the  first  half  and  the  structured  pic- 
torial method  in  the  last  half  of  the  training. 

In  each  method,  half  tjie  students  received  response-sensitive  feed- 
back, which  gives  both  the  correct  answer  and  the  reason  a wrong  answer 
is  wrong,  and  half  received  response-insensitive  feedback,  which  supplies 
only  the  correct  answer. 

Students  were  evenly  divided  across  all  conditions  by  their  general 
technical  (GT)  aptitude  score:  high  (at  or  above  124)  and  low  (below 
124)  . 

Posttests  on  visual  target  identification  and  verbal  target  cues 
evaluated  the  effectiveness  of  the  methods  and  feedback  conditions  for 
each  GT  level. 


Findings : 

Target  identification  performance  was  significantly  poorer  with  the 
structured  text  method  but  about  the  same  among  the  three  methods  using 
pictures.  Recognition  of  target  cues  was  significantly  better  when  ver- 
bal instructions  on  target  cues  were  given  than  when  training  was  en- 
tirely pictorial.  Type  of  feedback  had  no  significant  effect.  Learning 
performance  did  not  differ  as  a function  of  GT  aptitude,  but  interpreters 
with  lower  aptitude  forgot  their  training  more  rapidly. 
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utilization  of  Findings: 


r 

r 

i 


Effective  school  or  on-the-job  training  in  target  identification 
can  be  provided  with  a minimum  of  instructor  participation  and  relatively 
simple  support,  using  operational  imagery  as  the  basic  instructional 
material.  Immediate  feedback  on  right  or  wrong  answers  is  vital,  but  it 
need  not  be  complex. 
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BACKGROUND 

Theoretical  and  Practical  Aspects  of  Course  Design 

This  is  the  second  study  in  a series  exploring  instructional  tech- 
niques for  training  image  interpreters  in  target  identification.  A 
previous  study  (Root,  Gallagher,  & Sadacca,  1969),  using  computer- 
assisted  instruction  to  teach  interpreters  to  identify  U.S.  Army  cargo 
trucks  and  tanks,  e.nployed  two  main  variables;  the  method  of  presenting 
the  instructional  material  (linear  or  branching  format)  and  two  types  of 
interpreter  feedback  (response-sensitive  and  response-insensitive).  The 
linear  program  presented  the  instructional  material  in  a fixed  order, 
whereas  the  branching  program  permitted  the  more  able  students  to  skip 
over  material  that  was  too  easy.  Response-sensitive  feedback  provided 
the  students  with  information  appropriate  to  the  specific  answer  they 
had  given;  e.g.,  if  they  gave  wrong  answers,  the  -Feedback  explained  why 
the  answers  were  wrong.  Response-insensitive  ieedi.ack  provided  only  the 
correct  answer. 

The  first  study  indicated  that  the  branching  program  was  superior 
to  the  linear  program  in  that  less  time  was  required  to  train  interpret- 
ers to  a given  level  of  proficiency;  no  significant  differences  between 
response-sensitive  and  response-insensitive  feedback  were  shown.  Overall 
results  were  promising  and  indicated  that  some  form  of  programed  instruc- 
tion can  be  a very  effective  tool  for  upgrading  the  skill  level  of  image 
interpreters.  Nevertheless,  many  questions  remain  as  to  which  instruc- 
tional techniques  are  most  effective.  A very  important  consideration  is 
determining  the  type  of  program  that  most  effectively  upgrades  each  type 
of  skill. 

In  the  first  study,  the  identification,  or  discrimination,  skill 
was  taught  by  a fairly  conventional  method  that  relied  on  breaking  each 
target  type  down  into  significant  recognition  features,  which  the  stu- 
dents learn.  Research  evidence  as  to  the  effectiveness  of  this  approach 
is  not  clear  cut;  however,  some  evidence  suggests  it  is  not  necessary  to 
verbalize  the  difference  between  two  objects  to  make  a discrimination 
(Gibson,  1969) . Verbalization  may  be  an  artificial  and  unnecessary 
burden.  For  example,  an  interpreter  may  be  able  to  identify  an  M60  tank 
or  an  M48  tank  without  being  able  to  describe  its  characteristics  fully. 

Thus  there  is  a basic  theoretical,  yet  practical,  question  related 
to  course  design.  In  training  for  target  identification,  is  verbaliza- 
tion of  discriminating  characteristics  necessary,  or  does  it  inhibit 
learning?  From  a practical  viewpoint,  if  it  is  not  necessary  to  teach 
verbalization  of  discriminating  cues  or  features  of  a target,  instruc- 
tional material  can  be  developed  with  less  cost  and  less  careful  prepara- 
tion and  structuring. 
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Program  format  and  feedback  type  also  influence  the  cost  and  time 
of  developing  instructional  materials.  Branching  programs  cost  more  than 
linear  programs,  and  response-sensitive  programs  are  harder  to  prepare 
than  response-insensitive  ones.  Cost  is  also  affected  by  the  recurring 
need  to  revise  a program  each  time  new  elements  are  added  to  or  sub- 
tracted from  the  program.  A readily  modified  program  would  provide 
substantial  savings. 


Perceptual  Training 


The  research  background  for  the  experiment  cuts  across  several  major 
areas  studied  over  the  past  30  years.  Leibowitz  (1967)  discusses  much 
of  the  perceptual  research  relevant  to  image  interpretation,  relates  it 
to  interpreter  training,  and  provides  many  suggestions  for  further 
research  by  his  thorough  analysis  of  the  human  visual  system  and  image 
interpretation.  His  proposed  perceptual  teaching  method  is  the  most 
interesting  suggestion  for  the  present  experiment.  His  analysis  of  per- 
ceptual learning,  relying  mostly  on  the  work  of  Gibson  (1969) , suggests 
several  possible  ways  in  which  an  expert  image  interpreter  learns  to 
separate  relevant  from  irrelevant  cues  when  viewing  a complex  photograph. 
Two  of  the  most  appropriate  of  these  are  (a)  increase  in  specificity  and 
(b)  discovery  of  distinctive  features. 

Gibson  and  Gibson  (1953)  illustrated  "increase  in  specificity"  by  a 
nonsense  figure  or  "scribble."  Subjects  were  shown  a standard  figure 
and  then  asked  to  identify  that  figure  when  presented  with  a group  of 
figures  that  included  several  similar  scribbles.  Increases  in  the  number 
of  trials  produced  an  increase  in  specificity  of  recognition  of  the 
standard  figure,  as  shown  by  a decrease  in  the  number  of  drawings  which 
elicited  the  response  of  "same." 

These  subjects  (Gibson  & Gibson,  1953)  faced  a task  similar  to  that 
of  the  interpreter  in  attempting  to  differentiate  various  types  of  tar- 
gets. One  may  detect  an  object  as  a shapeless  spot,  recognize  it  as  a 
ship,  classify  it  as  a passenger  liner,  or  identify  it  es  the  Queen  Mary. 
The  role  of  verbalization  is  particularly  relevant  to  the  present  exper- 
iment. As  they  went  through  the  runs,  the  Gibson  subjects  made  more  and 
more  spontaneous  verbalizations  describing  the  differences  among  the 
scribbles.  The  subjects  were  never  taught  the  verbalizations,  and  some 
subjects  could  make  the  desired  discriminations  without  being  able  to 
put  verbal  "tags"  on  the  stimuli  to  describe  the  differences  among  them. 
The  investigators  concluded  that  verbalizations  were  not  the  cause  of 
the  increase  in  specificity,  but  merely  correlated  with  it — that  verbal- 
ization becomes  possible  as  one  discovers  new  dimensions  of  difference 
among  stimuli. 

Thus,  verbalizations  may  not  be  necessary  concomitants  of  skilled 
perceptual  acts.  This  is  not  to  imply  that  the  image  interpreter's 
verbalizations  are  not  meaningful,  but  rather  that  they  should  be 
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considered  of  limited  importance  because  they  do  not  necessarily  reveal 
the  complete  or  the  correct  basis  for  discriminations.  In  considering 
perceptual  learning,  it  is  helpful  to  consider  verbal  statements  regard- 
ing perceptual  acts  as  only  crftde  representations  of  the  breadth  and 
variety  of  cues  and  factors  involved  in  perception. 

A second  point  regarding  perceptual  learning  is  that  critical  iden- 
tifications are  based  on  the  distinctive  features  of  targets.  Gibson 
(1969)  points  out  that  a single  variable  or  property  rarely  determines 
the  uniqueness  of  an  object.  Rather,  it  is  a set  of  physical  character- 
istics that  permits  the  final  and  critical  identification.  Perceptual 
learning  in  such  instances  may  involve  the  interaction  of  more  cues  than 
the  individual  can  consciously  identify. 

Drawing  from  E.  J.  Gibson's  research,  Leibowitz  (1967)  contended 
that  the  present  military  methods  of  perceptual  training  may  suffer  from 
a number  of  deficiencies:  heavy  emphasis  on  verbal  concomitants,  failure 
to  exploit  the  critical  role  of  frequency  of  exposure,  and  no  attempt  to 
gradually  teach  recognition  of  distinctive  features.  Leibowitz  proposed, 
instead,  a perceptual  training  method  that  would  ta)<e  advantage  of  prc- 
gramed  instruction  technology.  In  this  method,  a large  series  of  images 
would  be  presented;  the  student  would  then  discriminate  among  them  and 
be  immediately  informed  of  the  correctness  of  his  choice.  Each  image 
would  emphasize  different  distinctive  features,  and  cues  to  the  correct 
answer  would  be  gradually  faded  as  the  student  demonstrated  progress. 

This  procedure  incorporates  a number  of  principles.  The  material 
is  not  taught  at  a verbal  level;  instead,  students  discover  for  them- 
selves the  features  that  provide  consistency  of  response.  Feedback  is 
informative;  it  helps  the  students  determine  from  the  complex  of  stimuli 
those  that  are  more  consistent  with  the  task  of  identifying  a certain 
type  of  target.  Student  attention  is  drawn  to  finer  and  finer  features 
corresponding  to  the  critical  details  used  by  the  skilled  interpreter. 
Perceptual  learning  is  accomplished  by  perceiving,  not  by  talking. 

In  image  interpretation,  it  is  important  that  interpreters  learn  to 
identify  targets  based  on  a wide  range  of  distinctive  cues,  because  tar- 
gets frequently  are  blurted  or  partially  hidden  by  trees  or  shadows. 
Interpreters  must  also  view  targets  at  various  scales,  orientations,  and 
attitudes,  and  in  different  configurations.  Therefore,  it  is  necessary 
to  determine  the  best  method  for  learning  the  wide  range  of  distinctive 
cues  associated  with  each  target. 

Leibowitz  argues  that  verbal  learning  of  visual  discriminations  is 
undesirable,  indeed  unnatural.  Verbalization  comes  after  discrimination, 
not  before,  and  generally  only  partially  covers  the  basis  for  the  dis- 
crimination. Perhaps  the  greatest  problem  in  learning  verbal  cues  is  the 
difficulty  of  finding  descriptive  words.  A cursory  inspection  of  dichot- 
omous keys  often  used  as  interpreter  aids  in  image  interpretation  reveals 
that  target  features  are  imperfectly  described;  trying  to  relate  such 
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features  as  actually  seen  on  the  imagery  to  the  verbal  descriptions  can 
be  very  frustrating.  Whole  paragraphs  would  be  needed  to  completely 
describe  features  such  as  the  shape  of  a turret.  It  is  hardly  reason- 
able to  expect  interpreters  to  memorize  paragraphs. 

Leibowitz  suggests  that  a better  method  for  perceptual  learning 
might  be  to  require  the  interpreter  to  make  visual  discriminations  and 
to  present  the  targets  in  a way  that  uses  all  possible  combinations  of 
distinctive  features.  Starting  with  the  easiest  discriminations  and 
gradually  working  down  to  fewer  and  fewer  cues  should  permit  teaching 
the  interpreter  to  identify  any  target  from  the  minimum  number  of  dis- 
tinctive cues. 


OBJECTIVE 

The  primary  purpose  of  the  present  study  was  to  develop  and  evaluate 
an  instructional  technique  for  training  image  interpreters  in  target 
identification  that  would 

1.  Be  easy  to  develop, 

2.  Require  only  materials  that  an  operational  unit  would  have 
readily  available, 

3.  Require  little  effort  for  updating  or  revision, 

4.  Be  flexible  in  terns  of  amount  of  training  that  could  be  pro- 
vided, ranging  from  a few  minutes  to  a few  hours,  and 

5.  Allow  free  choice  by  the  student  as  to  material  to  be  covered. 

Such  a flexible  and  simple  program  might  of  necessity  omit  certain  con- 
ventional features  generally  deemed  desirable.  Hence  material  designed 
to  teach  certain  points  would  not  be  included,  material  would  not  be 
ordered  according  to  difficulty,  and  special  material  for  feedback  pur- 
poses and  the  teaching  of  verbal  cues  would  be  omitted. 

To  demonstrate  the  potentialities  of  the  simplified  technique 
described  above,  three  other  instructional  programs  were  constructed. 
These  programs  included  certain  tenets  of  programed  instruction  and  were 
basically  designed  to  evaluate  the  need  for  verbalization  in  the  discrim- 
ination of  objects. 

Secondary  objectives  included  evaluation  of  the  relative  effective- 
ness of  two  competing  feedback  formats- -response-sensitive  and  response- 
insensitive — and  of  the  imp>ortance  of  the  trainee's  general  technical 
(GT)  aptitude  score  as  an  indicator  of  his  ability  to  learn  to  identify 
targets. 
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METHOD 


Instructional  Method 

Research  on  programed  instruction  has  been  extensive  but  has  pro- 
duced conflicting  findings.  However,  certain  concepts,  when  appropri- 
ately applied,  have  proved  beneficial  to  learning.  Feedback,  or  knowl- 
edge of  results,  has  been  shown  to  enhance  learning  (e.g.,  Stolurow, 

1968)  and  has  been  incorporated  into  all  four  training  methods.  Other 
research  has  indicated  that  such  conditions  as  low  error  rate  during 
learning,  structured  increase  in  the  difficulty  of  course  material, 
overt  response,  and  prompting  may  benefit  learning,  depending  on  the 
specific  application.  These  factors  were  considered  potentially  valu- 
able for  visual  discrimination  training  and  were  used  in  various  combi- 
nations in  the  three  comparison  training  methods. 

The  four  methods  studied  in  this  experiment  are  the  unstructured 
pictorial  method,  the  structured  textual  method,  the  structured  pic- 
torial method,  and  the  structured  method  (mixed) . 

Unstructured  Pictorial  Method.  In  the  general  case,  this  method 
entails  a data  bank  of  instructional  materials  and  a means  for  presenting 
these  materials  to  the  student.  The  presentation  means  could  be  a 
computer-controlled  display,  a teaching  machine,  a manually  operated 
slide  projector  controlled  by  the  student  or  an  instructor  or,  simply, 
the  scanning  of  hard-copy  photographs.  The  essential  features  are  that 
the  instructional  materials  do  not  require  elaborate  preparation  and  can 
be  presented  in  any  order  from  random  to  fixed,  while  new  materials  can 
be  added  or  deleted  without  affecting  operation  of  the  program.  In  the 
field,  operational  imagery  could  serve  as  the  instructional  material. 
Targets,  positively  identified  by  experienced  interpreters,  could  be 
annotated  on  selected  frames  of  imagery,  indexed,  and  stored  in  an  im- 
agery data  bank.  For  each  target,  feedoack  information  would  need  to  be 
developed  and  stored,  with  additional  feedback  in  the  form  of  keys  avail- 
able in  all  units  to  be  used  as  part  of  training.  This  is  the  instruc- 
tional technique  described  by  the  objective  of  the  study. 

For  this  experiment,  target  slides  with  no  identifications  and  with 
approximately  nine  targets  to  a slide  were  prepared,  together  with  key 
slides  showing  each  vehicle  at  four  different  scales.  All  vehicles  on 
key  slides  were  identified  on  the  slide. 

After  instructions  were  read  to  the  student,  the  key  slides  were 
presented  one  by  one  to  familiarize  the  student  with  the  target  types. 
Target  slides  were  then  presented  one  at  a time  in  a random  order,  and 
the  student  was  directed  to  identify  a specified  target  on  each  slide. 
After  each  response,  feedback  was  presented  as  follows: 

1.  For  the  response-sensitive  condition,  feedback  for  wrong  answers 
consisted  of  showing  the  student  the  correct  key  slide,  followed 
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in  a few  seconds  by  the  key  slide  for  his  incorrect  target. 

For  correct  answers,  feedback  involved  only  the  correct-answer 
key  slide.  (Later  in  the  course,  feedback  for  a correct  answer 
was  omitted  when  the  correct  answer  was  given  without  hesita- 
tion. ) 

2.  For  the  response-insensitive  condition,  the  correct-answer  key 
slide  was  shown  for  both  correct  and  incorrect  answers;  the 
incorrect-choice  slide  was  never  shown.  (Later  in  the  course, 
feedback  for  a correct  answer  was  omitted  when  the  correct 
answer  was  given  without  hesitation.) 

3.  Under  all  conditions,  appropriate  verbal  comments  were  made 
after  each  response;  e.g.,  for  the  response-sensitive  condition, 
the  comment  might  have  been  "Wrong,  the  truck  is  the  M37.  Slide 
23  shows  your  incorrect  choice,  the  M38  truck.  Notice  the  dif- 
ference between  these  two  trucks." 

The  second  half  of  the  course  followed  the  same  procedure  as  the 
first  half  but  employed  a different  set  of  slides  to  reduce  student 
tendency  to  memorize  backgrounds  rather  than  targets. 

Structured  Textual  Method.  This  method,  consisting  of  verbal  ques- 
tions and  multiple-choice  answers,  was  similar  to  the  usual  programed 
instruction  courses.  The  objective  was  to  teach  interpreters  to  memo- 
rize and  to  verbalize  the  cues  associated  with  each  tank  and  truck  by 
choosing  the  correct  multiple-choice  answer.  Cues  can  be  divided  into 
three  categories: 

1.  Cues  considered  essential  to  identification.  These  cues 
received  the  most  stress  (see  Figures  1 and  2). 

2.  Unique  cues  that  can  partially  or  completely  identify  a target. 
These  cues  received  intermediate  stress. 

3.  Cues  not  unique  to  a single  target  or  small  category  of  targets 
but  which  can  distinguish  among  larger  categories  of  targets. 
Such  cues  received  only  minor  stress. 

This  version  of  the  course  was  designed  to  teach  only  the  cues  to 
identification;  at  no  time  were  subjects  required  to  identify  the  tar- 
gets directly.  Response-sensitive  and  response-insensitive  feedback  for 
a correct  answer  was  given  by  presenting  the  correct  answer  (sometimes 
verbally  and  sometimes  both  verbally  and  on  a slide) . However,  for  an 
incorrect  answer,  response-sensitive  feedback  presented  the  correct 
answer  as  well  as  an  explanatory  comment  and  a specially  constructed 
feedback  slide  specific  to  the  particular  incorrect  answer. 

The  course  began  with  a slide  showing  all  trucks  or  tanks  to  be 
learned,  followed  by  a slide  of  a schematic  drawing  of  a tank  or  truck 
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Discrimination  aid:  schematic  drawing  of  a U.S.  cargo  truck. 


Figure  2.  Discrimination  aid:  schematic  drawing  of  a U.S.  tank. 


(Figures  1 and  2)  and  a series  of  questions  on  labeled  recognition 
features.  Next  came  a short  series  of  questions  on  the  dichotomous 
breakdowns  shown  in  Figures  3 and  4,  followed  by  a long  series  of  ques- 
tions on  specific  cues  (i.e.,  recognition  features).  At  first,  the  stu- 
dent could  easily  determine  the  correct  answer  merely  by  looking  at  the 
target  image  which  was  projected  on  the  screen.  For  example,  "What  is 
the  shape  of  the  fenders  on  the  M35  truck  shown  on  the  screen?"  Later 
the  target  image  was  omitted  for  similar  questions,  and  the  student  had 
to  answer  from  memory.  All  questions  were  repeated  until  the  student 
gave  the  correct  answer  readily. 

During  the  latter  part  of  the  course,  target  objects  on  prompting 
slides  became  progressively  more  difficult  to  see  because  of  degree  of 
concealment,  lighting  angle,  etc.,  and  therefore  were  less  and  less  help- 
ful in  answering  questions.  This  procedure  insured  that  the  students 
trained  by  this  method  and  by  the  other  methods  used  slides  of  the  same 
difficulty.  For  this  method,  both  training  and  key  slides  had  the  iden- 
tification printed  just  beneath  the  picture  of  each  vehicle. 

Structured  Pictorial  Method.  This  method  was  essentially  the  same 
as  the  unstructured  pictorial  method  except  that  it  included  the  spe- 
cially constructed  training  and  feedback  slides  (described  under 
structured  textual  method)  and  an  enforced  low  error  rate.  To  maintain 
a low  error  rate,  the  data  bank  was  structured  into  difficulty  levels, 
and  slides  were  presented  in  sets  of  slowly  increasing  difficulty,  from 
the  easiest  to  the  hardest.  All  slides  in  each  set  had  to  be  learned 
and  readily  identified  before  the  next  set  started. 

Feedback  was  essentially  the  same  as  that  for  the  unstructured 
pictorial  method  except  that  response-sensitive  feedback  for  each 
incorrect  answer  used  special  key  slides  that  showed  a photograph  of 
the  correct  target  and  the  incorrect  one  side-by-side  for  quick  compar- 
ison. , For  the  unstructured  pictorial  method,  this  comparison  was  shown 
with  two  slides  in  succession. 

Structured  Method  (Mixed) . This  method  was  identical  to  the  struc- 
tured textual  method  for  the  first  half  of  the  instructional  period. 
During  the  second  half  of  each  course,  students  identified  target  exam- 
ples as  they  did  in  the  structured  pictorial  method.  In  the  second  half, 
identification  difficulty  started  at  an  intermediate  level  and  progressed 
by  greater  increments  than  under  the  structured  pictorial  method,  to  in- 
sure that  students  were  exposed  to  targets  just  as  difficult  as  in  the 
other  methods. 


Experimental  Design 

A factorial  design  was  used  to  provide  an  empirical  evaluation  of 
(a)  the  four  training  methods,  (b)  feedback  type  (response-sensitive 
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Figure  4.  Discrimination  aid:  recognition  features  for  a set  of  U.S.  trucks. 


versus  response-insensitive) , and  (c)  GT  aptitude  level  (high  versus 
low) . Figure  5 shows  the  experimental  design  and  Table  1 the  analysis- 
of- variance  layout. 


Subjects 

Subjects  were  32  enlisted  men,  recent  graduates  of  the  image  inter- 
preter course  at  the  U.S.  Army  Intelligence  School,  Fort  Holabird,  Md. 
These  men  were  familiar  with  identification  of  Soviet  equipment  but  were 
inexperienced  in  identification  of  U.S.  Ammy  equipment.  Each  man  was 
assigned  to  one  of  the  four  training  methods  and  one  of  the  feedback  con- 
ditions. Each  was  classified  according  to  GT  score — high  (124  and  above) 
or  low  (123  or  below).  Equal  numbers  of  enlisted  personnel  with  high  and 
low  GT  scores  were  assigned  to  each  of  the  eight  experimental  groups. 


Stimulus  Materials 


Imagery  Slides.  To  provide  the  flexibility  required  in  con- 
structing training  materials,  realistic,  plasi^ic,  scale  models  of  U.S. 
trucks  and  tanks  were  photographed  against  a r^listic  terrain  model. 
Variations  in  photographic  parameters  (scale,  shadow,  camera  angle, 
concealment,  and  resolution)  maintained  fidelity  of  simulation.  Trees 
and  shadows  to  conceal  cues  on  some  of  the  images  forced  the  student 
to  attend  to  less  dominant  cues.  Use  of  various  lighting  angles 
changed  contrast  and  emphasized  selected  features,  and  the  blurring  of 
some  of  the  images  introduced  resolution  changes.  Feedback  slides, 
specially  constructed  for  response-sensitive  feedback  conditions,  per- 
mitted all  possible  combinations  of  target  types  taken  two  at  a time 
to  be  photographed  side-by-side  for  direct  comparison. 

Slides  used  were  70mm  by  100mm.  Each  course  required  two  ca- 
rousels, each  containing  60  slides.  For  the  three  structured  courses, 
each  training  slide  was  divided  into  four  quadrants  with  one  target  in 
each.  Typical  instructions  for  a slide  would  be  "Identify  the  tank 
shown  at  45B,"  or  "Notice  the  tank  shown  at  46C."  For  the  unstructured 
pictorial  method,  each  slide  contained  approximately  nine  targets,  with 
each  target  numbered.  For  these  slides,  a typical  instruction  might  be 
"Identify  the  truck  shown  at  annotation  6,  slide  45." 

In  the  120  slides  available  for  each  method,  the  number  of  train- 
ing targets  and  key  slides  were  as  follows: 

1.  Unstructured  pictorial  method:  900  training  targets,  20-24 
keys, 

2.  Structured  textual  method:  200  training  targets,  70-100  keys. 
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Experimental  design. 


Table  1 


Source  of  Variance  and  Degrees  of  Freedom  for  Analysis  of  Variance 


Between  subjects 

Training  methods  (T) 

Feedback  type  (F) 

GT  aptitude  (A) 

TF 

TA 

FA 

TFA 

Residual  = e. 


Total 

Within  subjects 

Order 

Content 

OT 

OF 

OA 

CT 

CF 

CA 

Residual  = e_ 


Total 


32 


3.  Structured  pictorial  method:  300  training  targets,  50-50 


keys,  and 

4.  Structured  method  (mixed):  250  training  targets,  60-80  keys. 

Feedback . Response-sensitive  and  response-insensitive  feedback 
provided  a basic  experimental  comparison.  For  response-sensitive 
experimental  conditions,  feedback  for  wrong  answers  consisted  of  exam- 
ples of  both  the  correct  and  incorrect  targets,  along  with  a verbal 
explanation  of  the  error;  for  a correct  answer,  only  an  example  of  the 
correct  target  was  presented.  For  response-insensitive  conditions,  an 
example  of  the  correct  target  was  shown  for  both  correct  and  incorrect 
answers,  accompanied  by  an  appropriate  verbal  statement  from  the  exper- 
imenter. (Under  both  conditions,  the  feedback  example  for  a correct 
answer  was  omitted  later  in  the  course  when  a correct  answer  was  given 
without  hesitation.) 


Proficiency  Tests 


Pretests  and  posttests  were  developed  for  measuring  change  in 
identification  performance  as  a function  of  training.  Each  test  simul- 
taneously assessed  target  identification  performance  for  both  tanks  and 
trucks.  Slides  for  the  pretest  were  presented  in  random  order,-  those 
for  the  posttest  were  shown  in  an  ascending  order  of  difficulty  (diffi- 
culty ratings  based  on  expert  image  interpreter  judgments) . Difficulty 
varied  as  a function  of  degree  of  concealment,  lighting  angle,  image 
scale,  and  so  forth.  The  tank  images  were  divided  about  equally  between 
firing  position  and  travel  position;  truck  images  showed  canvas  covers 
in  place  and  removed  in  about  equal  numbers.  The  tests  contained  200 
targets — 98  tanks  and  102  trucks. 

In  addition  to  the  target  identification  pretest  and  posttest,  a 
verbal  paper-and-pencil  posttest  was  constructed,  consisting  of  100 
multiple-choice  questions  (50  for  tanks  and  50  for  trucks)  edsout  the 
cues  associated  with  each  tank  and  truck. 


Experimental  Equipment  Configuration 

Slides  were  rear-projected  by  a random  access  slide  projector 
(TELEPRO  RA60)  controlled  through  an  IBM  1092  programed  keyboard.^ 


Commercial  or  trade  names  are  given  only  in  the  interest  of  precision 
in  reporting  experimental  procedures.  Use  of  the  names  does  not  con- 
stitute official  endorsement  by  the  Army  or  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 
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The  experimenter  could  call  up  any  slide  by  pressing  the  appropriate 
keyboard  buttons.  During  each  session,  four  projectors  trained  four 
subjects,  with  each  experimental  booth  screened  from  the  others.  The 
size  of  the  projected  image  was  large  enough  to  permit  subjects  to  sit 
back  from  the  screen  and  identify  the  targets,  although  subjects  were 
allowed  to  move  as  close  to  the  screen  as  they  desired. 

Each  station  was  controlled  by  one  experimental  assistant,  who  was 
equipped  with  a schedule  showing  slide  presentation  order,  4-in.  x 6-in. 
question  cards,  4-in.  x 6-in.  answer  cards,  forms  for  recording  respon- 
ses, and  computational  aids  to  determine  the  verbal  feedback  and  the 
feedback  slides  to  be  presented.  The  original  plan  called  for  computer 
control  of  laboratory  equipment,  with  training  and  feedback  slides  pre- 
sented automatically.  However,  because  the  computer  progrcim  was  not 
available  at  the  time  of  this  study,  the  computer's  role  was  simulated 
by  experimental  assistants  at  each  student  station. 

Administration 

Each  group  of  four  subjects  required  1-1/2  days  to  complete  the 
experiment.  After  receiving  general  instructions  from  the  experimenter, 
each  subject  was  assigned  to  an  experimental  station,  and  the  pretest 
started.  Pretest  scores  confirmed  that  the  subjects  had  no  familiarity 
with  U.S.  Army  vehicles.  As  soon  as  the  experimental  assistant  deter- 
mined that  the  subjects  were  performing  at  a chance  level,  the  pretest 
was  stopped.  Normally,  this  occurred  after  about  30  targets  had  been 
presented  (about  10  minutes) . 

Each  of  the  two  courses  (tank  or  truck)  was  designed  to  take  3 
hours,  broken  into  four  sessions  of  45  minutes  each.  At  the  start  of 
the  first  45-minute  session,  each  assistant  read  to  the  students  the 
instructions  for  the  course  to  be  used  and  began  asking  them  questions 
or  showing  them  slides  with  targets  to  be  identified. 

After  the  first  course  was  completed,  the  first  45-minute  session 
of  the  second  course  was  begun.  The  first  experimental  day  ended  with 
completion  of  the  second  45-minute  session  of  the  second  course. 

During  the  morning  of  the  second  experimental  day,  the  two  remain- 
ing 45-minute  sessions  were  completed  and  the  posttest  was  started. 

After  students  completed  the  target  identification  posttest,  they  were 
given  the  verbal  posttest. 


Dependent  Variables 


The  primary  dependent  variable  was  the  percentage  of  targets  iden- 
tified correctly  on  the  posttest.  Subscores  included  these  values: 
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1.  Percentage  of  tanks  identified  correctly, 

2.  Percentage  of  trucks  identified  correctly,  and 

3.  Percentage  of  all  vehicles  identified  correctly. 

The  score  for  the  verbal  posttest  was  the  percentage  of  questions 
answered  correctly  for  tanks,  for  trucks,  and  for  tanks  and  trucks 
combined. 


RESULTS 

The  results  for  the  main  effects  for  both  posttests  appear  in 
Table  2;  they  show  that  the  structured  textual  method  gave  poorer 
identification  performance  than  any  of  the  other  three  methods.  Table  3 
summarizes  the  analysis  of  variance  for  these  scores.  The  training- 
methods  variable  yielded  mean  differences  significant  at  the  .01  level 
of  confidence.  The  results  indicate  that  methods  permitting  students 
to  develop  their  own  discrimination  cues  were  more  effective  than  those 
requiring  the  student  to  learn  specified  cues.  However,  the  results 
show  no  sizable  performance  differences  among  the  three  remaining  meth- 
ods, which  indicates  that  an  unstructured  presentation  of  available 
operational  imagery  with  minimal  (i.e.,  response-insensitive)  feedback 
can  serve  as  a valuable  tool  in  target  identification  training  at  least 
as  well  as  more  structured  presentations. 

The  structured  method  (mixed)  resulted  in  target  identification 
performance  equivalent  to  that  of  the  unstructured  pictorial  method  and 
the  structured  pictorial  method,  and  equivalent  to  the  structured  tex- 
tual method  on  the  verbal  test  designed  to  determine  the  subject's 
knowledge  of  target  recognition  features  (cues) . These  results  indi- 
cate that,  for  interpreters  to  be  able  to  verbalize  recognition  features 
(e.g.,  for  the  development  of  operational  keys  or  for  on-the-job 
training) , a minimum  of  structured  training  on  such  cues  is  probably 
required.  The  structured  method  (mixed)  used  only  half  the  verbal 
materials  of  the  structured  textual  method,  yet  it  resulted  in  equiv- 
alent performance.  However,  the  structuring  of  such  training  materials 
will  require  much  time  and  effort. 

A method  modeled  after  the  unstructured  pictorial  method  can  pro- 
vide effective  school  or  on-the-job  training  for  target  identification 
with  minimal  instructor  participation  and  relatively  simple  support 
materials.  If  opaque  imagery  is  used,  response-insensitive  feedback  may 
be  provided  by  indicating  the  correct  identification  on  the  back  of  the 
print.  The  only  obligatory  structure  is  in  the  instructional  setting: 
a situation  that  will  insure  interpreters  will  concentrate  on  the  learn- 
ing situation.  Failure  to  monitor  student  behavior  may  reduce  the  po- 
tential value  of  this  very  unstructured  training  method.  If  a computer 
is  available  to  monitor  the  instruction,  personnel  can  be  released  for 
other  duties. 
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Table  2 


Mean  Percentage  of  Correct  Scores  on 
Identification  and  Verbal  Posttests 


Target  Identification  Test 

Total  Total  Total  Verbal 

Main  variable  tanks  trucks  vehicles  test 


Training  method: 


Unstructured  pictorial 

69.3 

79.9 

74.6 

59.1 

Structured  textual 

57.1 

64.0 

60.5 

84.9 

Structured  pictorial 

71.8 

75.7 

73.8 

65.9 

Structured  (mixed) 

70.1 

79.5 

74.8 

84.4 

Feedback  type : 

Response-sensitive 

70.7 

75.6 

73.2 

76.1 

Response-insensitive 

63.5 

73.9 

68.7 

71.0 

General  technical 
aptitude : 

Above  median  (GT  ^ 124) 

67.8 

76.6 

72.2 

74.4 

Belo-w  median  (GT  S 123) 

66.8 

72.9 

69.6 

72.8 

Order: 

Studied  first 

Studied  second 

70.0 

71.8 

72.2 

74.9 

Content-: 


Tanks 

Trucks 


67.1 


74.8 


67.1 

74.8 


71.5 

75.6 


Source 

df 

MS 

F 

Between  subjects 

Training  method 

(T) 

3 

773.43 

6.05** 

Feedback  type 

(F) 

1 

317.73 

2.49 

GT  aptitude 

(A) 

1 

106.60 

.83 

TF 

3 

85.96 

.67 

TA 

3 

186.70 

1.46 

FA 

1 

4.85 

.04 

TFA 

3 

65.00 

.51 

Residual  = e^ 

16 

127.74 

Subtotal 

31 

Within  subjects 

Order 

(0) 

1 

53.65 

1.06 

Content 

(C) 

1 

947.10 

18.72** 

CT 

3 

47.45 

.94 

OF 

1 

59.30 

1.17 

OA 

1 

259.22 

5.12* 

CT 

3 

34.65 

.68 

CF 

1 

123.21 

2.44 

CA 

1 

20.26 

.40 

Residual  = e^ 

20 

50.58 

Subtotal 

32 

Total 

63 

*p  ^ .05. 

**p  < .01. 
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Table  4 shows  the  results  of  each  training  method  by  feedback  type 
and  GT  aptitude.  There  is  a slight  but  nonsignificant  tendency  for 
j response-sensitive  feedback  to  yield  better  learning  and  a general  but 

I not  totally  consistent  tendency  for  higher  aptitude  subjects  to  perform 

better. 

I 

i 

j Table  4 

1 

j Mean  Percentage  of  Correct  Target  Identifications: 

I Training  Method  by  Feedback  Type  and  GT  Aptitude 


Training  method 

Varicibles 

Unstructured 

pictorial 

Structured  Structured  Structured 
textual  pictorial  (mixed) 

Feedback  type : 


Response-sensitive 

80.3 

61.6 

75.0 

75.7 

Response- insensitive 

68.9 

59.4 

72.5 

74.0 

GT  aptitude: 

Above  median  (GT  ^ 124) 

71.2 

65.3 

76.2 

76.3 

Below  median  (GT  < 123) 

78.0 

55.8 

71.4 

73.4 

Table  5 presents  the  scores  made  on  the  paper-and-pencil  posttest, 
and  Table  6 shows  the  analysis  of  variance  for  these  scores.  Differ- 
ences in  performance  due  to  training  method  were  statistically  signif- 
icant (p  < .01).  Subjects  trained  with  target  cue  materials — structured 
textual  method  and  structured  method  (mixed) — scored  highest. 


CONCLUSIONS  AND  IMPLICATIONS 

Results  generally  seem  to  support  Leibowitz's  contention  that 
training  image  interpreters  to  rely  on  the  learning  of  specific  verbal 
cues  for  target  identification  may  not  be  a particularly  effective 
method.  A more  effective  method  seems  to  be  the  teaching  of  target 
identification  by  showing  the  interpreter  a large  number  of  images  and 
requiring  an  identification,  followed  by  feedback  on  whether  the  iden- 
tification was  correct. 
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Table  5 


j1 

r 

! 


Mean  Percentage  Scores  on 

Verbal  Posttests  by  Target  Type 

Target 

class 

Variables 

Tanks 

Trucks 

Mean 

Training  methods: 

Unstructured  pictorial 

57.5 

60.8 

59.1  'j 

Structured  textual 

82.0 

87.8 

84.9 

Structured  pictorial 

64.5 

67.2 

65 . 9 

Structured  (mixed) 

82.0 

86.8 

84 . 4 

Feedback  type : 

• 

Response- sensitive 

73.5 

78.8 

76 . 1 

Response- insensitive 

69.5 

72.5 

71 . 0 

GT  aptitude : 

Above  median  (GT  ^ 124) 

72.5 

76.^ 

74 . 4 

Below  median  (GT  < 123) 

70.5 

75.0 

72.8 

i 


i 


Table  6 


Analysis  of  Variance  for  Verbal  Scores  ] 

Made  on  Posttests  j 


Source 

df 

ms 

F 

Between  subjects 

Training  methods 

(T) 

3 

2732.92 

10.45** 

Feedback  type 

(F) 

1 

420.25 

1.61 

GT  aptitude 

(A) 

1 

42.25 

.16 

TF 

3 

412.25 

1.58 

TA 

3 

66.92 

.26 

FA 

1 

30.25 

.12 

TFA 

3 

90.75 

.12 

Residual  = e^^ 

16 

261.50 

Subtotal 

31 

Within  subjects 

Order 

(0) 

1 

109.75 

1.60 

Content 

(C) 

1 

272.25 

3.98 

OT 

3 

220.91 

3.23* 

OF 

1 

6.25 

.09 

OA 

1 

110.25 

1.61 

CT 

3 

7.58 

.11 

CF 

1 

20.25 

.29 

CA 

1 

2.25 

.03 

Residual  = e^ 

20 

68.45 

Subtotal 

32 

Total 

63 

*n  < rm 
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